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PREFACE 



In the University of California and California State University segments, pure and applied research are recognized, along 
with teaching, as primary missions. These institutions are ranked among the best in the world for the creation of new 
knowledge and for solving some of our most pressing human problems. On the other hand, the mission of the community 
colleges is to provide quality teaching and an environment conducive to learning to most of the State's freshmen and 
sophomore students. The emphasis placed on learning is reflected in the high degree of faculty contact with students 
both in and out of the classroom. Although community college teaching loads are the highest of the three segments and 
the students are the most diverse in terms of incoming preparedness, ethnic background, socioeconomic status, and age, 
California cOiinmunity colleges have developed creative programs and innovative methods of instruction to serve our ever- 
changing client^.le. 

While it may seem that there is no room for research in a segment devoted almost exclusively to teaching and learning, 
community colleges, out of necessity, are taking the lead in a form of research known as institutional research. For many 
years, our colleges have suffered because policy makers were making intuitive judgments about our programs and 
students. Their conclus^ Dns were, ai times, incorrect but convenient for a particular philosophy or preconceived notion. 
Their actions were usually unsupported by any real evaluation data. However, we did not have much data or careful 
research to show that our programs and services were effective and efficient. In an article in the Sacramento Bee, in March 
of 1984, Pat Callan, former direccor of the California Postsecondary Higher Education Commission warned that if we 
resist evaluating our programs, we face evaluation by outsiders. In this environment, institutional research is given new 
emphasis in community colleges. 

While opportunities and resources are limited for community college faculty to participate in academic research , the very 
nature of our mission and the diversity of our students provide limitless opportunities for practical rest^rch on teaching 
effectiveness. Classroom-based research, as advocated by Dr. K. Patricia Cross, now Professor of Higher Education at 
the Ui uversity of California at Berkeley, is currently being done in our segment, and I expect that we will see more emphasis 
on classroom research as faculty members evaluate new methods of delivery, new ways to encourage critical thirVing, 
and other attempts to develop literacy skills throughout the curricula. 

As a community college chief executive officer, I see a definite need for accurate, timely information and incisive analyses. 
Over time, the decisions made by governing boards and administrators have dramatic effects on our colleges and the kinds 
of programs and services provided to our students. With tne right information and proper planning, we can shape the 
future for our institutions rather than just meeting it. 

In this first in a series of Research Commission papers, the authors preient a compelling case for institutional research 
as it relates to decision making in California's community colleges. They define it, they discuss its uses to support decision- 
making and policy formulation, they cite its importance in the evaluation and improvement of programs and services to 
keep our institutions oriented to serve the needs of today's students and our changing communities, they point out how 
it can be done; they even list examples of questions decision-makers should be asking of institutional researchers. On 
behalf of the Commission, I thank the authors for a clearly written and uselul document. I hope that our governing boards, 
chief executive officers, and other college decision-makers turn to it to reaffirm the importance and usefulness of 
institutional research in California's community colleges. 



George R. Boggs, Ph.D. 
Superintendent/President 
Palomar Community College District 
Chair, Commission on Research 
California Association of Community 
Colleges 
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EXECUTIVE SQivlMARY 



Quality Indicators. Student Outcomes. Measures of Accountability. Evaluation Criteria. Per- 
formance-Based Funding Factors. 

Tne effort to measure educational effectiveness is surfacing as a major new focus for higher education. 
Increasingly, local constituencies and state interests are asking community colleges to demonstrate that the 
programs and services l»iey provide work. 

This political reality of accountability is affecting all community colleges. Districts, colleges, and the commu- 
nity college system as a whole, at a minimum, must be able to answer these crucial questions: 

• What are the important demographic, economic, and social trends in the state and in the 
communities that will be affecting community colleges? What will be the nature of the colleges* 
communities in the future? Who will be the potential students? 

• What kinds of students are enrolled currently in the colleges? Do they differ by college? Do these 
students reflect the colleges* communities at large? 

• What kinds of preparation do students bring to community colleges? Are they prepared for 
college-level classes or could theu benefit from remediation? 

e What are the educational goals of students? Do these goals differ by age, gender, ethnicity, work 
status, or economic level? 

• How well are students* goals being met? How well do community college programs and services 
work? Is the evidence solid? 

• What do students do after they leave the coll6:je? Are they successful as transfers to four-year 
institutions? In finding jobs? In improving their skills and potential if currently employed? 

• What are the implications for colleges of the differences in student" demographics and goals? 

• How can institutional program and service effectiveness be improved? 

Through institutional research these and other questions can be answered. Institutional decision-making 
includes identifying areas of concern, problem-solving, establishing missions, goals and objectives, policy- 
making, and developing short and long range institutional plans. Institutional research identifies and gathers 
information to assist decision makers. Data are extracted, analyzed, summarized and communicated in an 
accessible manner. Institutional research encompasses al least six areas: information systems use, program 
evaluation, survey design and implementation, research end analysis, research critiques, and emerging issue 
formulation. 

In a system as diverse as the California community colleges, institutional research is conducted in a variety 
of ways by a variety of people in a variety of positions. Often institutional research may be conducted by 
the very person who raises the question or who has a problem that needs solving, or responsibility may 
be assigned to a staff member as a full-time professional pursuit, or as a partial reassignment from other 
duties. 
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The institutional research fur :uon can be organized and perfonned in several ways. Many organizational 
fonns are being used successfully in California community colleges. However, there are three basic forms: 
the institutional research and planning office, the institutional research committee, and individual and group 
research. No matter how it is accomplished, people conducting institutional research play a key role for 
decision makers in gathering, integrating and analyzing data. They can also play a leadership role in detecting 
emerging external issues and forces that will be shaping their college's future and hence enable the college 
to respond to those forces Ln a more proactive manner than otherwise. They are most 3ffective when their 
reports are timely, readable, organized and identify implications for policy and procedures. 

Questions related to educational and institutional effectiveness will continue to be the major focus of decision- 
making about California's community college system through the 1990's. Continued support from the many 
communities and constituencies served or wishing to be served by our institutions requires that community 
colleges demonstrate convincingly that their programs and services are effective. This, in turn, requires an 
effective institutional research capability in our colleges and districts, and the system as a whole. 
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ACCOGNTABILITY FOR COMMUNITY COLLEGES 



Quality Indicators. Student Outcomes. Measures of Accountability. Evaluation Criteria. 
Performance-based Funding Factors. 

Whatever it is called, the concern for measuring educational effectiveness is surfacing as a major new focus 
for higher education, in general, and community colleges, in particular. Increasingly, local consutue. Jes and 
state interests are asking community colleges to demonstrate that the programs and services they provide 
work. 

While arguments about what constitutes appropriate measures abound, one thing is clear: legislative and 
public support for educational programs and services cannot be assumed. Legislatures and the public are 
demanding documented evidence as a condition of continued support. Unfortunately, there is a scarcity of 
meaningful, policy-oriented research on community college institutional or program effectiveness at the state 
level, and while some solid research exists at the local level, it has yet to be assembled into a cohesive body 
of knowledge that legislatures and colleges find useful and convincing. 

Across the nation, the concem for student outcomes manifests itself in report after report. Repeatedly, 
educational institutions at all levels are being called upon to demonstrate quality in their educational practices! 
Community colleges are being asked to track students from admission and registration, through college 
programs and services, and on into employment or transfer to baccalaureate granting institutions, and to 
report their findings to the public, the legislature, and other govemmental agencies. As community colleges 
strengthen their capabilities to accomplish these and other studies-known collectively as institutional research 
—they can expect to accrue both extemal and intemal benefits. 

Institutional research provides extemal benefits by demonstrating to the general public, the legislature, and 
to others at the state level, that community college programs are effective, that students and society alike 
benefit from them, and that they are a wise expenditure of public funds. Indeed, had California's community 
colleges been able to do this during the past six years, it is likely there would have been far less legislative and 
public criticism— and far more money. 

Institutional research provides intemal benefits by assessing which teaching strategies work best with each 
particular type of student, which programs are well coordinated with those at high schools and four-year in- 
stitutions, and whether, for example, assessment and remediation activities and programs are having the 
desired effect of improving student performance. 

CRUCIAL QUESTIONS 

The new era of accountability means that districts, colleges, and the community ccllege system as a whole, 
must bt able to answer a series of crucial questions about their students, programs, and communities. The 
major questions are these: 

• What are the important demographic, economic, and social trends in the state and in the com- 
munities that will be affecting community colleges? What will be the nature of the colleges* 
communities in the future? Who will be the potential students? 

• What kinds of students are enrolled cun-ently in the colleges? Do they differ by college? Do these 
students reflect the colleges* communities at large? 

• What kinds of preparation do students bring to community colleges? Are they prepared for college- 
level classes or could they benefit from remediation? 
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What are the educational goals of students? Do these goals differ by age, gender, ethnicity, woik 
status, or economic level? 

How well are students* goals being met? How well do community college programs and services 
work? Is the evidence solid? 



• What do students do after they leave the community college? Are they successful as transfers to 
four-year institutions? In finding jobs? In improving their skills and potential if currently employed? 

• What are the implications for colleges of the differences in student demographics and goals? 

e How can institutional program and sersnce effectiveness be improved? 

California community colleges must be able to answer these student and institutional success questions, as well 
as questions related to finances, if colleges and districts, and the system as a whole, are to respond effectively 
to legislative, state agency, and community questions about their success in carrying out their mission. 
Fortunately, much of the data needed to answer these and other questions resides in college and district 
computers. However, frequently the ability to extract and analyze the data and then to summarize the data 
and transform them into information easily accessible and useful to decision-makers is less well-developed. 

in. TRUSTEE DECISION-MAKING AND INSTITUTIONAL RESEARCH 

Decisions. Fiscal Decisions. Policy Decisions. Program Decisions. 

Trustees must make hard decisions based primarily upon recommendations of chief executive officers and 
other district and college staff. Institutional research analyses and studies can provide and enhance the foun- 
dation upon which decisions are made. Recommendations can then be based on information about the 
potential desired and dysfunctional outcomes, as well as underlying rationales and costs. 

While all decision-makers can benefit from the information identilied by the crucial questions outlined in the 
previous section, for proo«^Ti-sp€cific decisions, decision-makers usually require additional information; in- 
formation provided by a program needs assessment. A typical program needs assessment answers the fol- 
lowing questions: 
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^ Is there a distinct community need for the program or project that would best be filled by the 
college? Information may show that, while there is not a broad community need, there is a need 
to serve a particular population— perhaps a population that has been under-served. 

• Are there alternatives to the program or project and what are the relative costs of these alterna- 
tives? Have other community colleges or institutions tried similar projects and what were the 
results? 

• What effect will a proposed project or program have on special constituencies in the district, such 
as business and industry, ethnic populations, women, residents of specific geographical areas and 
other private or governmental institutions? 

• How docs the program or project fit into the long range plan for the college? 

• If the decision involves use of college facilities, what are the competing needs for these facilities? 

• If the decision involves a curriculum change, how will various student subpopulations be affecteu? 

Community college trustees are charged with the task of making informed policy decisions which affect the 
lives of their constituents— the community students faculty and staff. Such decisions are best made when 
based on accurate, informed research. 
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SCOPE OF INSTITUTIONAL RESEARCH for DECISION-MAKING 



Institutional decision-making includes problem-solving, establishing missions, goals and objectives, policy- 
making, and developing short and long rang<> institutional plans. Institutional research identifies and gathers 
information about needs, solutions, implementations, and possible problems. It anticipates and analyzes 
consequences, develops conceptual perspectives, and scans the environment. It provides reports on these 
activities in concise, timely, and understandable fomi in order to assist in decision-making, problem-solving 
and planning. Consequently, a key function of institutional research is to provide decision-making information 
at every stage. Often, institutional research widens the range of options considered and challenges "taken- 
for-qranted assumptions" about goals, processes, and outcomes. 

Institutional research encompasses at least six areas: information systems use, program evaluation, survey 
design and implementation, research and analysis, research critique, and fomiulation of emerging issues. 
Underlying these six activities of institutional research is the organization of infomiation for decision-makers 
in a practical, timely, and comprehensible fashion for program and institutional evaluation and planning. 

Information systems use provides the data foundation for institutional research. Ideally, a college's infor- 
mation systems contain data cn all cun-ent activities within the college and provide an historical record as well. 
Institutional research sorts, selects, and presents data that are useful for planning and evaluation. Via 
information systems, institutional research can make projections and evaluations and support planning 
processes. 

Program evaluation is a research process that enhances institutional effectiveness by providing feedback 
on program performance. This feedback can be used either by those external to the program to enable 
informed judgments about whether and how to support the program or by those internal to the program to 
help those responsible to shape its development. Data from various sources are often reviewed during pro- 
gram evaluation. Quantitative, qualitative and perceptual information are used, since these different types of 
information provide different perspectives enabling a more accurate and responsive evaluation. 

Survey design and implementation is another important activity of institutional research. Survey research 
detemiines the views of clients and potential clients of college programs and serwces. Surveys usually 
supplement data collected in routine transactions with students and provide information on those not 
attending the college. General surveys of students add depth to a college's understanding of who its students 
are; individually-focussed surveys provide in-depth infomiation on specific activities. 

The research and analysis activity of institutional research may be its most critical. Analytical and research 
studies acquire quantified and conceptual infomiation about the subject. This information is analyzed and 
assessed as a foundation for further study and as a basis for interpreting findings. With this context even '^Tiited 
available data may prove helpful to decision -makers. A well-established, comprehensive institutional research 
program can provide analyses on many educational issues such as how students flow through the college's 
courses and programs or how programs are most effectively reviewed. Institutional research can also provide 
feasibility and impact assessments. 

The research critique activity of institutional research involves analyzing and assessing the value of research 
and impact of policy papers published by a variety of state agencies, research organizations, and educational 
institutions. Local colleges are faced with an increasing need to react insightfully to key external reports, such 
as A Nation at Ri<;k, that comment on their policies and practices. 

The formulation of emerging issues results from collaborative processes of systematically scanning for 
new opportunities and identifying shifts induced by external forces. When emerging issues or forces are iden- 
tified early, then colleges have wide scope for proactive response. When important issues or forces are not 
detected early, colleges can only react with a nan-ow range of responses. Certain colleges in CaHfomla, for 
example, defected the emergence of state mandated matriculation years before its arrival and are now 
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providing model piograms f ^ the remaining colleges. Deteding, assessing, and linking new opportunities and 
forces enables a thoughtful assessment of their significance. Leadership may respond through the pilot testing 
of new strategies. For example, community colleges are taking the lead in learning outcomes absessment and 
classroom research to improve learning and teaching. 

Institutional research uses information systems and specific studies in a Complementary fasnion. This enables 
the college to be responsive 'o a broad spectrum of decision-making and planning needs and to benefit from 
reviewing several possible decision options. 

V. ORGANIZATION OF INSTITaTIONAL RESEARCH 

In a system as diverse as the k^lifomia community colleges, institutional research is conducted in a variety 
of ways by a variety of people in a variety of positions. Often institutional research is done by the very person 
who raises the question or who has a problem that needs solving. Faculty members, deans of student services, 
deans Oi instruction, personnel directors, and administrators, as well as professional institutional researchers 
can all be involved in one or more institutional research activities at community colleges. The amount of time 
each of these individuals devotes to research also varies considerably. For some it is a full-time professional 
pursuit. Others have a significant amount of release time. Still others try to fit institutional research activities 
into an already busy work schedule. 

The institutional research function can be organized and penormed in several ways. Many organizational 
forms are being used successfully in Califomia community colleges. However, there are these three basic 
forms: 

Institutionad Research and Planning Office: This office has the designated function of organizing, 
conducting,and facilitating college or district research and planning. Generally the office reports to the 
president or a vice president. 

Institutional Research Committee: A formal research committee with several staff members shares the 
responsibility for institutional research. The committee usua!!^ identifiers rasearch topics and divides the tasks 
among various individuals or offices. 

Individual or Group Research: In this model, research iS conducted on specific topics of interest by inter- 
ested staff members. For example, an English professor or faculty group might review the success rate of 
students in se eral levels of English composition courses. 

Whichever model is used, comprehensive and quality institutiofial research requires time, competent person- 
nel, and resources. 

VL SOME QUALITIES OF AN INSTITUTIONAL RESEARCHER 

Frequently, community collects employ a specialist in institutional research or designate a person as the chief 
institutional researches Institutional researchers at Califomia community colleges come from many profes- 
sional backgrounds. Some are trained in research techniques, others have become researchers as a result of 
the!. Ixcen interest in education and a desire to improve educational institutions and programs. All require the 
development of good skills in statistics, evaluation techniques, and communication skills. An institutional 
researcher mu?t be able to work cooperatively with others. Good interpersonal and organizational skills are 
necessary hs'w^se research is a complex group effort. An effective researcher works easily with facu'.y, 
admin.^trators and board members to idctify their research need:, to design and implement appropriate 
sKriies, 'Aud to communicate the results in an accessible manner. 

In providiny their services, instL ,tional researchers aim for certain key study f>jid report attributes that affect 
the ultimate usefulness of their wori^. These include; 
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Timeliness: Research reports and data analyses summaries which are out-of-sync with the 
institution's decision processes are often perceived as having little value by local decision makers 
and even as wasting scarce institutional research resources. To be useful, institutional research 
repoits and summaries of the results c' data analyses must reach decision-makers before decisions 
are made. 

Readability: Institutional research reports and data analyses which rely on technical language 
and statistically elegant but esoteric discussions may be seen as complicating rather than clarifying 
appropriate directions for action. To be useful, institutional research reports and summaries of 
data analyses must be readable and written in language that is accessible to those who participate 
in the decision-making process. 

Organization: Sometimes excellent studies and data analys3s are lost in reports and infonna- 
tion summaries which either are poorly organized or which do not make apparent the relation- 
ship of their findings to the decisions to be made. To be useful, institutional research reports and 
summaries of data analyses must be organized in ways that enable decision-makers to find the 
inform^*ion they need in the form they need it. 

Policy and Procedure Relationship: Research reports and data analyses results which fail to 
make specify the relations between findings and institutional policy and procedure considerations 
usually are regarded by decision-makers as "nice-to-know" but not really helpful. To be useful, 
institutional research reports and data analyses musi identify implications for institutional policies 
and procedures. As appropriate, they also identify directions for change. 

IMPLICATIONS FOR THE FUTURE 

Institutional research can be a dynamic and vital component of the decision-making processes of California's 
community college districts, their colleges, and the system as a whole. In addition, institutional research can 
provide an information and analytical base for enhancing the support from communities served and from 
governmental and other agencies which provide the funding and coordination vital to the delivery of essential 
programs and services. 

To accomplish these objectives, institutional researchers must focus on presenting the results of their studies 
and analyses effectively. In turn, establishing an organizational structure to support institutional research is 
necessary if the college wishes to obtain quality decision support. Institutional research becomes particularly 
effective when its reports and analyses identify implications for instructional policies and procedures, and 
suggest, where applicable, appropriate changes or altematives. With the detection of emerging issues, 
institutional research can even enable an institution to become a proactive educational leader. 

Questions related to educational and institutional effectiveness will continue to be the major focus of decision 
making about California's community colbge system through the I990*s. Continued support from the many 
communities and constituencies served or wishing to be served by our institutions requires that community 
colleges demonstrate convincingly that their programs and services are effective. This, in turn, requires an 
effective institutional research capability in our colleges and districts, and the system as a whole. 



BIBLIOGRAPHY 



Commission for Educational Quality, Southern Regional Education Board. "Access to Quality Undergraduate 
Education/' Chronicle of Higher Education. July 3, 1985, pp. 9-12. 

Commission for the Review of the Master Plan for Higher Education. The ChaUenge of Change: A 
Reassessment of the California Community CoUeges. Sacramento: State of California, 1986. 

Commission on Instruction, California Association of Community Colleges. Mission. Finance, Accounta- 
bility: A Framework for Improvement. Sacramento: California Association of Community Colleges 
Press, 1984. 

Cox Coffey, Janis. "Proving What We are Doing is Working: The Student Flow Research Model." ACCCA 
Management Report. 1986 - 7/5. 

Education Commission of the States. "Transforming the State Role in Undergraduate Education. Time for 
a Different View." The News. California Association of Community Colleges, 1986, 32(1), pp. 4-9. 

Joint Committee for the Review of the Master Plan for Higher Education, California Legislature. California 
Community College Reform. Sacramento: State of California, 1986. 

Study Group on the Conditions of Excellence in American Higher Education. Involvement in Learning: 
Realizing the Potential of American Higher Education. Washington, D.C.: National Institute of 
Education, 1984. 



ERLC 



14 

8 



CALIFORNIA ASSOCIATION OF COMMGNITY COLLEGES 



NAME — 

TITLE 

COLLEGE 
ADDRESS 



SGBSCRIPTION ORDER 

COMMUNITY COLLEGE ISSUES, VOLUME 1 (6 ISSUES) 
NUMBER OF SUBSCRIPTION DESIRED $30.00 = $. 



PAYMENT ENCLOSED PLEASE BILL 



INDIVIDGAL ISSGE ORDER 
COMMGNITY COLLEGE ISSGES, VOLGME 1 NGMBER 6 

INSTITUTIONAL RESEARCH FOR DECISION-MAKING AND ACCOUNTABILITY IN 
CAUFORNIA COMMUNITY COLLEGES 

NUMBER OF COPIES DESIRED 

(@ $6.00 EACH FOR 1-5 COPIES. @ $5.00 EACH FOR 6 OR MORE COPIES) 

TOTAL COST $ (NUMBER OF COPIES X $6.00 OR $5.00) 

PAYMENT ENCLOSED PLEASE BILL 



SIGNATURE 



Please make check payable to CACC PRESS, and return with completed order form to: 

COMMUNITY COLLEGE ISSUES 
CACCPRESS 

CAUFORNIA ASSOCIATION OF COMMUNITY COLLEGES 
2017 "O" STREET 
SACRAMENTO, CA 95814 



ERIC 



15 



CALIFORNIA ASSOCIATION OF COzMMGNITY COLLEGES 



OFFICEx^S: 

Patricia G. Siever, District Academic Senate President, Los Angeles 

Community College District, President 
Roberta Mason, Member, Board of Trustees, Lake Tahoe Community 

College District^ Vice-President 
Lois A. Callahani President, College of San Mateo, Past President 

DIRECTORS: 

Linda Allen, Student Trustee, Compton Community College District 
Peter Blomerley, President and Superintendent, Fremont-Newark 

Community College District 
Barbara Davis, Professor, Social Science Division, Sacramento City College 
John Feare, Professor of Counseling, Grossmont College 
Hilary Hsu, Chancellor and Superintendent, San Francisco Community 

College District 

Melvina Jones, Professor of European History, Antelope Valley College 
Diana Lockard, Member, Board of Trustees, Citrus Community College District 
David Lopez-Lee, Member, Board of Trustees, Los Angeles Community 
College District 

Peter MacDougall, President and Superintendent, Santa Barbara 

Community College District 
Mary Reynaga, Student Trustee, Palomar Community College District 
Herbert Sussman, President and Superintendent, Rio Hondo Community 

College District 

STAFF: 

Peler M. Hirsch, Executive Director 

Cindra J. Smith, Associate Executive Director 

Walter C. Rilliet, Commissioner of Athletics 

Rita Mize, Director of Governmental Affairs 

Jennifer Robertson, Director of Public Information Services 

Ron Richardson, Administrative Assistant to the Commissioner of Athletics 

Denise Henrikson, Manager of Fiscal Services 

Martha E. Mathias, Secretary to the Executive Director and to the Board of Directors 
Debra A. Stephen, Secretary to the Commissioner of Athletics 
Kim Meister, Secretary to the Associate Executive Director 
Martha Christiansen, Secretary 
Leslie Leder, Receptionist/Secretary 



ERIC Clearinghouse for 

j-J^r> i6 Junior Colleges 

JAN 2 0 1989 



